We propose a method of dimensioning and managing the bandwidth of a link on which flows with heterogeneous access-link bandwidths are aggregated. We use a processor-sharing queue model to develop a formula approximating the mean TCP file-transfer time of flows on an access link in such a situation. This only requires the bandwidth of the access link carrying the flows on which we are focusing and the bandwidth and utilization of the aggregation link, each of which is easy to set or measure. We then extend the approximation to handle various factors affecting actual TCP behavior, such as the round-trip time and restrictions other than the access-link bandwidth and the congestion of the aggregation link. To do this, we define the virtual access-link bandwidth as the file-transfer speed of a flow when the utilization of the aggregation link is negligibly small. We apply the virtual access-link bandwidth in our approximation to estimate the TCP performance of a flow with increasing utilization of the aggregation link. This method of estimation is used as the basis for a method of dimensioning the bandwidth of a link such that the TCP performance is maintained, and for a method of managing the bandwidth by comparing the measured link utilization with an estimated threshold indicating degradation of the TCP performance. The accuracy of the estimates produced by our method is estimated through both computer simulation and actual measurement.

The number of users of video on demand (VoD) services has increased dramatically. In VoD services, the demand for content items changes greatly hour to hour. Because service providers are required to maintain a stable service during peak hours, they need to design system resources based on the demand at peak time, so reducing the server load at this time is important. Although multicast delivery, in which multiple users requesting the same content item are supported by one delivery session, is effective for suppressing the server load during peak hours, user response times can increase greatly. A peer-to-peer-assisted delivery system, in which users download content items from other users watching the same content item, is also effective for reducing server load. However, system performance depends on selfish user behavior, and optimizing the usage of system resources is difficult. Moreover, complex operation, i.e., switching the delivery multicast tree or source peers, is necessary to support video cassette recorder (VCR) operation, e.g., fast forward, rewind, and pause. In this paper, we propose to reduce server load without increasing user response time by multicasting popular content items to all users independent of actual requests as well as providing on-demand unicast delivery. Through a numerical evaluation that uses actual VoD access log data, we clarify the effectiveness of the proposed method.

Analysing and modeling of traffic play a vital role in designing and controlling of networks effectively. To construct a practical traffic model that can be used for various networks, it is necessary to characterize aggregated traffic and user traffic. This paper investigates these characteristics and their relationship. Our analyses are based on a huge number of packet traces from five different networks on the Internet. We found that: (1) marginal distributions of aggregated traffic fluctuations follow positively skewed (non-Gaussian) distributions, which leads to the existence of "spikes", where spikes correspond to an extremely large value of momentary throughput, (2) the amount of user traffic in a unit of time has a wide range of variability, and (3) flows within spikes are more likely to be "elephant flows", where an elephant flow is an IP flow with a high volume of traffic. These findings are useful in constructing a practical and realistic Internet traffic model.

We investigate the impact of traffic on the performance of large-scale NAT (LSN), since it has been attracting attention as a means of better utilizing the limited number of global IPv4 addresses. We focus on the number of active flows because they drive up the LSN memory requirements in two ways; more flows must be held in LSN memory, and more global IPv4 addresses must be prepared. Through traffic measurement data analysis, we found that more than 1% of hosts generated more than 100 TCP flows or 486 UDP flows at the same time, and on average, there were 1.43-3.99 active TCP flows per host, when the inactive timer used to clear the flow state from a flow table was set to 15 s. When the timer is changed from 15 s to 10 min, the number of active flows increases more than tenfold. We also investigate how to reduce the above impact on LSN in terms of saving memory space and accommodating more users for each global IPv4 address. We show that to save memory space, regulating network anomalies can reduce the number of active TCP flows on an LSN by a maximum of 48.3% and by 29.6% on average. We also discuss the applicability of a batch flow-arrival model for estimating the variation in the number of active flows, when taking into account that the variation is needed to prepare an appropriate memory space. One way to allow each global IPv4 address to accommodate more users is to better utilize destination IP address information when mapping a source IP address from a private address to a global IPv4 address. This can effectively reduce the required number of global IPv4 addresses by 85.9% for TCP traffic and 91.9% for UDP traffic on average.

The increased performance of mobile terminals has made it feasible to collect data using users' terminals. By making the best use of the network performance data widely collected in this way, network operators should deeply understand the current network conditions, identify the performance-degraded components in the network, and estimate the degree of their performance degradation. For their demands, one powerful solution with such end-to-end data measured by users' terminals is network tomography. Meanwhile, with the advance of network virtualization by software-defined networking, routing is dynamically changed due to congestion or other factors, and each end-to-end measurement flow collected from users may pass through different paths between even the same origin-destination node pair. Therefore, it is difficult and costly to identify through which path each measurement flow has passed, so it is also difficult to naively apply conventional network tomography to such networks where the measurement paths cannot be uniquely determined. We propose a novel network tomography for the networks with undeterministic routing where the measurement flows pass through multiple paths in spite of the origin-destination node pair being the same. The basic idea of our method is to introduce routing probability in accordance with the aggregated information of measurement flows. We present two algorithms and evaluate their performances by comparing them with algorithms of conventional tomography using determined routing information. Moreover, we verify that the proposed algorithms are applicable to a more practical network.

We develop a method of dimensioning and managing the bandwidth of a link on which TCP flows from access links are aggregated. To do this, we extend the application of the processor-sharing queue model to TCP performance evaluation by using flow statistics. To handle various factors that affect actual TCP behavior, such as round-trip time, window-size, and restrictions other than access-link bandwidth, we extend the model by replacing the access-link bandwidth with the actual file-transfer speed of a flow when the aggregation link is not congested. We only use the number of active flows and the link utilization to estimate the file-transfer speed. Unlike previous studies, the extended model based on the actual transfer speed does not require any assumptions/predeterminations about file-size, packet-size, and round-trip times, etc. Using the extended model, we predict the TCP performance when the link utilization increases. We also show a method of dimensioning the bandwidth needed to maintain TCP performance. We show the effectiveness of our method through simulation analysis.

In this paper, we propose a method of detecting TCP performance degradation using only bottleneck-link utilization statistics: mean and variance. The variance of link utilization normally increases as the mean link-utilization increases. However, because link-utilization has a maximum of 100%, as the mean approaches 100%, the possible range of fluctuation becomes narrow and the variance decreases to zero. In this paper, using the M/G/R processor sharing model, we relate this phenomenon to the behavior of flows. We also show that by using this relationship, we can detect TCP performance degradation using the mean and variance of link utilization. In particular, this method enables a network operator to determine whether or not the degradation originates from the congestion of his/her own network. Because our method requires us to measure only link utilization, the cost of performance management can be greatly decreased compared with the conventional method, which requires dedicated functions for directly measuring the TCP performance.

A method is described that can allocate bandwidth to each user flow fairly in a scalable network architecture such as differentiated services architecture. As promising queueing techniques for providing differentiated services, class-based packet scheduling and selective packet discarding have been attracting attention. However, if we consider that bandwidth should be allocated to each flow in a weighted manner, the parameters used in these methods such as the weight assigned to each class queue should be pre-determined appropriately based on an assumption about the number of flows in each class. Thus, when the actual traffic pattern differs from the assumed one, they may not work well. Instead of assuming the traffic conditions, our method estimates the number of active flows in each class by simple traffic measurement and dynamically changes the weight assigned to each class queue based on the estimated number. Our method does not need to maintain the per-flow state, which gives it scalability. Simulation showed that this method is effective under various patterns of the number of active flows.

Machine learning (ML) has been used for various tasks in network operations in recent years. However, since the scale of networks has grown and the amount of data generated has increased, it has been increasingly difficult for network operators to conduct their tasks with a single server using ML. Thus, ML with edge-cloud cooperation has been attracting attention for efficiently processing and analyzing a large amount of data. In the edge-cloud cooperation setting, although transmission latency, bandwidth congestion, and accuracy of tasks using ML depend on the load balance of processing data with edge servers and a cloud server in edge-cloud cooperation, the relationship is too complex to estimate. In this paper, we focus on monitoring anomalous traffic as an example of ML tasks for network operations and formulate transmission latency, bandwidth congestion, and the accuracy of the task with edge-cloud cooperation considering the ratio of the amount of data preprocessed in edge servers to that in a cloud server. Moreover, we formulate an optimization problem under constraints for transmission latency and bandwidth congestion to select the proper ratio by using our formulation. By solving our optimization problem, the optimal load balance between edge servers and a cloud server can be selected, and the accuracy of anomalous traffic monitoring can be estimated. Our formulation and optimization framework can be used for other ML tasks by considering the generating distribution of data and the type of an ML model. In accordance with our formulation, we simulated the optimal load balance of edge-cloud cooperation in a topology that mimicked a Japanese network and conducted an anomalous traffic detection experiment by using real traffic data to compare the estimated accuracy based on our formulation and the actual accuracy based on the experiment.

With the advent of high speed links, online flow measurement for, e.g., flow round trip time (RTT), has become difficult due to the enormous demands placed on computational resources. Most existing measurement methods are designed to count the numbers of flows or sizes of flows, but we address the flow RTT measurement, which is an important QoS metric for network management and cannot be measured with existing measurement methods. We first adapt a standard Bloom Filter (BF) for the flow RTT distribution estimation. However, due to the existence of multipath routing and Syn flooding attacks, the standard BF does not perform well. We further design the double-deletion bloom filter (DDBF) scheme, which alleviates potential hash collisions of the standard BF by explicitly deleting used records and implicitly deleting out-of-date records. Because of these double deletion operations, the DDBF accurately estimates the RTT distribution of TCP flows with limited memory space, even with the appearance of multipath routing and Syn flooding attacks. Theoretical analysis indicates that the DDBF scheme achieves a higher accuracy with a constant and smaller amount of memory compared with the standard BF. In addition, we validate our scheme using real traces and demonstrate significant memory-savings without degrading accuracy.

We propose an algorithm for finding heavy hitters in terms of cardinality (the number of distinct items in a set) in massive traffic data using a small amount of memory. Examples of such cardinality heavy-hitters are hosts that send large numbers of flows, or hosts that communicate with large numbers of other hosts. Finding these hosts is crucial to the provision of good communication quality because they significantly affect the communications of other hosts via either malicious activities such as worm scans, spam distribution, or botnet control or normal activities such as being a member of a flash crowd or performing peer-to-peer (P2P) communication. To precisely determine the cardinality of a host we need tables of previously seen items for each host (e.g., flow tables for every host) and this may infeasible for a high-speed environment with a massive amount of traffic. In this paper, we use a cardinality estimation algorithm that does not require these tables but needs only a little information called the cardinality summary. This is made possible by relaxing the goal from exact counting to estimation of cardinality. In addition, we propose an algorithm that does not need to maintain the cardinality summary for each host, but only for partitioned addresses of a host. As a result, the required number of tables can be significantly decreased. We evaluated our algorithm using actual backbone traffic data to find the heavy-hitters in the number of flows and estimate the number of these flows. We found that while the accuracy degraded when estimating for hosts with few flows, the algorithm could accurately find the top-100 hosts in terms of the number of flows using a limited-sized memory. In addition, we found that the number of tables required to achieve a pre-defined accuracy increased logarithmically with respect to the total number of hosts, which indicates that our method is applicable for large traffic data for a very large number of hosts. We also introduce an application of our algorithm to anomaly detection. With actual traffic data, our method could successfully detect a sudden network scan.